Papers with KV-cache compression
Exploring Fine-Tuning for In-Context Retrieval and Efficient KV-Caching in Long-Context Language Models (2026.eacl-short)
Copied to clipboard
| Challenge: | Long-Context Language Models (LCLMs) can encode entire document collections, offering a strong alternative to retrieval-augmented generation (RAG). |
| Approach: | They propose to use LCLMs to encode documents with context windows of millions of tokens to improve their performance. |
| Outcome: | The proposed training strategies improve long-context performance and their robustness under compression techniques. |
OjaKV: Context-Aware Online Low-Rank KV Cache Compression (2026.findings-acl)
Copied to clipboard
Yuxuan Zhu, David H. Yang, Mohammad Mohammadi Amiri, Keerthiram Murugesan, Tejaswini Pedapati, Pin-Yu Chen
| Challenge: | Existing methods for inference use static, offline-learned subspaces that perform poorly under distribution shifts. |
| Approach: | They propose a framework that integrates a storage policy with an online subspace adaptation to preserve key-value tokens in full rank as high-fidelity anchors. |
| Outcome: | Experiments show that OjaKV maintains or improves zero-shot accuracy at high compression ratios, achieving the strongest gains on long-context benchmarks requiring complex reasoning. |